Dimension Reduction Using Principal Component Analysis for Pharmaceutical Domain

N Deepa, Chandrasekar Ravi

 

ABSTRACT:

Huge number of researchers work in engineering, remote sensing, agriculture, economics, biology domains now-a- days. The major challenges they face are huge number of observations and simulations they collect for their research work. Advanced techniques in data collection and storage capabilities lead to data overload. Due to the increase in the number of observations, some statistical methods fail partially. The dimension of data is nothing but the number of input variables in each observation. One of the present Mathematical challenges is the high dimensional dataset used for research purpose. Researcher may not be able to predict the importance of the variables measured on each observation. We need a mathematical technique which can reduce the dimension of the data. There are many statistical data reduction techniques currently in use viz., Singular Value Decomposition (SVD), Mahalanobis Taguchi Method, Principal Component Analysis, Factor Analysis. Among these, Principal Component Analysis is Multivariate statistical unsupervised dimension reduction technique most popularly used now-a-days because of its simplicity. In this paper, a sample dataset has been taken and by applying Principal Component analysis, dimension of the data has been reduced. The results of MATLAB program written for Principal Component Analysis by applying the sample dataset is given in this paper.

 

INTRODUCTION:

From past few years, many of the insurance companies have collected a huge amount of data in their data storage. Some of this information are known to be redundant because it is also used of other purposes. And also there are many sources of external data available from suppliers. These external sources may not be familiar with the data in the data sources. This huge amount of data is made available to create predictive or classification model. This amount of information which has been collected from various multiple sources leads to creation of numerous variables for building a predictive

or classification model.  When such modeling project involves numerous variables, first step to be done is to reduce the number of variables. Since there is multitude of variables, it is difficult to find out the relationship between variables. Numerous variables reduce the efficiency of predictive and classification models. In order to reduce the number of variables to a subset of variable clusters, clustering algorithms can be used to improve efficiency. Variable clustering algorithm provides groups of variables where variables in a group are similar to the variables in the same group and are dissimilar to variables belong to other groups¹. Principal components are continuous solution of the cluster membership indicators in K-means clustering algorithm². A geometrically motivated algorithm for representing the high- dimensional data was proposed which provides a computationally efficient approach to nonlinear dimensionality reduction that has locality-preserving properties and a natural connection to clustering³. Supervised principal components is another type of principal component similar to conventional principal components analysis except that it uses a subset of the predictors selected based on their association with the outcome. Supervised principal components can be applied to regression and generalized regression problems, such as survival analysis⁴. The steps for calculating principal component analysis are explained with an example⁵.

 

Problem Definition

Principal component analysis is a variable reduction procedure. It is useful when you have obtained data on a number of variables (possibly a large number of variables), and believe that there is some redundancy in those variables. In this case, redundancy means that some of the variables are correlated with one another, possibly because they are measuring the same construct. Because of this redundancy, you believe that it should be possible to reduce the observed variables into a smaller number of principal components (artificial variables) that will account for most of the variance in the observed variables. The applications of multivariate data analysis to tablet compression were aimed at understanding of the connection between process variables and final product properties^6-8. Principal component analysis was applied to in-die force and displacement data and ranked materials in terms of their relative plasticity⁹. Principal component analysis was explored to be a useful tool in addressing the multidimensional complexity of the deformation behaviors of pharmaceutical raw materials¹⁰. Therefore Principal component analysis can be used for dimension reduction in any Pharmaceutical product manufacturing.

 

MATERIALS AND METHODS:

Factor analysis is otherwise called an statistical dimension reduction method which is used to depict between the pragmatic and interrelated variables in provosts for a lower quantity of unnoticed variables is called factors. The modeling of the observed variables is done with the help of potential factors and error terms. The variables in dataset can be reduced with the collected information of interdependencies of the observed variables. By the observation we can say that this technique is equal to the low rank approximation of the matrix. The factor analysis is classifieds into two types Exploratory factor analysis and Confirmatory factor analysis. By using the regression modeling technique we can test the error terms which is related to principal component analysis (PCA).The intricate interrelationships among the items and set of the items can be recognized by using the technique of Exploratory factor analysis(EFA).

When compared with EFA the Confirmatory factor analysis (CFA) is a more intricate approach. The structural modeling of the equation is tested by using CFA. This model can hold the dimension of the error and it is less preventive than the least- squares. The models of hypothesis can be tested by using the actual data and loadings of the observed variables.

 

The method of factor extraction is done with the help of PCA which is the first stage of the EFA. By computing the factor weights we can mine the possible variance at the max and this model is scrutinized.

 

There are number of basis to recommend that factor analysis is similar to PCA. From our observation we can say that the PCA is faster with the less number of resources compared with the factor analysis⁵. The results produced by these methods more are less related with each other. The Researchers can gain the more information from a PCA approach, such as an individual’s gain on a assured component. So, the results of PCA are an version of the large amount of variation of experimental variables. The FA relation for universal dissent in the facts. By adjusting the correlation matrix we can insert the diagonals of it. The amount of squared perpendicular distance of the component axis can be minimized by using the PCA. FA approximation factors which persuade retort on observed variable. In PCA the module scores of a linear grouping of the observed variables prejudiced by Eigen vectors. In PCA the components which do not signify the original constructs but it can be interpreted by using the exact model stipulation.

 

Steps used to perform Principal Component Analysis are as follows¹¹:

 

The input to PCA is a dataset, in matrix A, comprised of n variables (columns) collected from m observations (rows).

 

Step 1: Data standardization

As discussed above, this step is performed in order to avoid dealing with variables measured by different units. The mean and standard deviation are computed for each variable (column). The mean in each column, is reduced from every value in that column and the result is divided by the corresponding standard deviation.

 

Step 2: Computing correlations between the variables

This step computes the correlation coefficient between each pair of variables in the dataset. Computing the correlation coefficient between each pair of unstandardized variables is identical to computing the covariance between each pair of standardized variables. Once the covariance measures have been computed for each pair of variables, they are stored in a matrix (covariance matrix).

 

Step 3: The connection between the data, the covariance matrix, its eigen values, its eigenvectors, the principal components, and computing all of them.

 

As explained above, PCA searches for the directions of the principal components. These directions are vectors generate by performing a linear transformation on the data, originally spanned by n, correlated to a certain degree, variables, into n uncorrelated new variables. However, since PCA aims at reducing dimensionality, only k < n of the new variables will be used eventually to present the data. The transformation used is linear combination on the n original variables, thus the objective of this step is to obtain the linear combinations which directions constitute correlation trend lines in the data

 

Step 4: Projecting the data on the reduced space spanned by the principal components.

PCA eventually reduces the dimensions of the data according to the number of principal components that cover a sufficient amount of variation in it. In addition, the orientation of the data is rotated such that the directions of the principal components correspond to the axes of the coordinate system presenting the data. Computationally, this is achieved by pre-multiplying the transposed score matrix with a transformation matrix. This transformation matrix is the matrix that consists of the eigenvectors of the covariance matrix, arranged in rows in descending order of the corresponding eigen values. The number of eigenvectors in the transformation matrix determines the number of dimensions the data will be transformed into.

 

Implementation Methodology

Matlab program has been written to implement Principal Component analysis. Here x is input matrix consists of variables in columns and observations in rows.

[r,c]=size(x);

m=mean(x);

for i=1:r

    for j=1:c

        xmean(i,j)=x(i,j)-m(j);

    end

end

c=corrcoef(x)

[pc,v]=eig(c)

%extract diagonal

v=diag(v);

[row,col]=size(pc);

%sort in descending order

[j,rind]=sort(-1*v);

v=v(rind);

pc=pc(:,rind);

signals=xmean*pc'

cumper=cumsum(v)./sum(v);

cumper*100

for k=1:4

v1(k)=var(signals(:,k));

end

v1

cumper1=100*v1/sum(v1)

[ro,co]=size(signals);

corrcoef(signals)

figure;

x=[1:4];

plot(1:1:4,v,'--mo','LineWidth',2,...

                'MarkerEdgeColor','k',...

                'MarkerFaceColor',[.49 1 .63],...

                'MarkerSize',5);

title('Screen Plot of Principal Component analysis  4 variables');

xlabel('Number');

ylabel('Eigen Values');

figure;

plot(1:32,signals(:,1),1:32,signals(:,2),1:32,signals(:,3),1:32,signals(:,4));

legend({'1st Component' '2nd Component' '3rd Component' '4th Component'},  ...

    'location','NW');

 

Principal component analysis involves a mathematical procedure that transforms a number of (possibly) correlated variables into a (smaller) number of uncorrelated variables called principal components. The first principal component accounts for as much of the variability in the data as possible, and each succeeding component accounts for as much of the remaining variability as possible.

 

By using the correlation matrix of the data we construct the compute the eigenvectors in Principal Component Analysis. The fraction of confliction of the unique data can be re-builded by using the Eigen values. Likewise, the foremost few eigenvectors can frequently be construed in provisos of the large-scale physical deeds of the structure. By spanning the few eigenvectors we can very well reduce the original space.

 

The first component extracted in a principal component analysis accounts for a maximal amount of total variance in the observed variables. Under typical conditions, this means that the first component will be correlated with at least some of the observed variables. It may be correlated with many.

 

The second component extracted will have two important characteristics. First, this component will account for a maximal amount of variance in the data set that was not accounted for by the first component. Again under typical conditions, this means that the second component will be correlated with some of the observed variables that did not display strong correlations with component 1.

 

The second characteristic of the second component is that it will be uncorrelated with the first component. Literally, if you were to compute the correlation between components 1 and 2, that correlation would be zero. The remaining components that are extracted in the analysis display the same two characteristics: each component accounts for a maximal amount of variance in the observed variables that was not accounted for by the preceding components, and is uncorrelated with all of the preceding components. A principal component analysis proceeds in this fashion, with each new component accounting for progressively smaller and smaller amounts of variance (this is why only the first few components are usually retained and interpreted). When the analysis is complete, the resulting components will display varying degrees of correlation with the observed variables, but are completely uncorrelated with one another¹².

 

RESULTS AND DISCUSSIONS:

Table 1. Sample Observations

 

In the field of Principal Component Analysis, the numerical procedure which uses the orthogonal transformation to adapt a set of observations of almost certainly related variables into a set of values of linearly uncorrelated variables is called as principal components. The amount of principal components is fewer than or identical to the number of unique variables.

 

The various steps used in Principal component analysis are:

Step 1: Get some sample observations (Table 1)

Step 2: Subtract the mean from each of the dimensions.

For the sample observations , the mean subtracted result is given below in Table 2.

 

Table 2. Mean subtracted result

Step 3: Covariance matrix has to be calculated

The result of covariance matrix for the given observations is as follows:-

 

Covariance Matrix =

1.0000	0.4234	0.2176	-0.3363
0.4234	1.0000	-0.2113	-0.4356
0.2176	-0.2113	1.0000	-0.7135
-0.3363	-0.4356	-0.7135	1.0000

 

Step 4: Eigenvectors and Eigenvalues of the covariance matrix have to be found

For the sample observations, here are the eigenvectors and eigenvalues:

 

Eigenvectors                 =                 

-0.4687	-0.3246	0.8113	-0.1295
-0.3720	-0.6815	-0.4113	0.4775
-0.4746	0.6359	0.0766	0.6038
0.6456	-0.1609	0.4083	0.6250

 

EigenValues                  =                 

2.0197	0.0000	0.0000	0.0000
0.0000	1.2960	0.0000	0.0000
0.0000	0.0000	0.6366	0.0000
0.0000	0.0000	0.0000	0.0477

 

Step 5: Principal components and feature vectors have to be calculated

From the above step, calculated eigen vectors and eigen values are used for choosing components. It should be noted that eigenvector with the highest eigen value is the principal component of the given data set. The next step is to order the eigen vectors by eigen value from highest to lowest. We need to choose only first few eigen vectors so that the final data set will have only p dimenstions. Next feature vector is constructed by forming a matrix with the eigenvectors in the columns.

 

FeatureVector =(eig1, eig2, eig3, …..eign)

 

Step 6: Deriving the new data set

After choosing the components, take the transpose of the vector and it need to be multiplied with the left of the original data set that has been transposed.

 

FinalData = RowFeatureVector X RowDataAdjust

Where RowFeatureVector  is the matrix with the eigenvectors in the columns transposed so that the eigenvectors are now in the rows, with the most significant eigenvector at the top, and RowDataAdjust  is the mean-adjusted data transposed, ie. the data items are in each column, with each row holding a separate dimension. FinalData is the final data set with data items in columns and dimensions along rows. FinalData  is given in Table 3.

 

 

 

Table 3. Principle components

 

The number of principal components to be retained depend upon the availability of variables for further analysis. The final data ie. Principal components are uncorrelated to each other.

 

CONCLUSION:

Principal component analysis could be used as a tool in regression analysis, clustering or classification problems because it is basically a dimension reduction technique as it often shows that most of the variability in the data can be explained by the first few principal components. So a good characterization of the data can be seen in lower dimensional spaces. Projection of the data on the first two principal components can be very useful in identifying characteristics in the data. It is basically an exploratory data analysis tool. The Principal Component analysis is fretful with clearing up the variance-covariance configuration of a set of variables. The purpose of PCA is data diminution ie., by moving the linear combinations original variables to few composite variables. These fresh variables called as principal components which is a combination of composite variables and original variables. In this paper, it has been explained with an example how innovative variables are altered to composite variables which is nothing but Principal components.

 

REFERENCES:

1.     Kriegel, H. P et. al. A General Framework for Increasing the Robustness of PCA-Based Correlation Clustering Algorithms. Scientific and Statistical Database Management. Lecture Notes in Computer Science. 2008.

2.     C. Ding and X. He. K-means Clustering via Principal Component Analysis. Proc. of Int'l Conf. Machine Learning (ICML 2004), pp 225–232. July 2004.

3.     M. Belkin and P. Niyogi. Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation, 15(6);2003:1373-1396.

4.     Bair Eric et. al. Prediction by Supervised Principal Components. Journal of the American Statistical Association 101. 473(19);2006:119-137.

5.     N Deepa et al. Variable Reduction Techniques – A comparative study. IJRIT International Journal of Research in Information Technology. 1(10);2013:19-23.

6.     Haware RV et. al. Application of multivariate methods to compression behavior evaluation of directly compressible materials. Eur J Pharm Biopharm. 72(1);2009:148–55.

7.     Haware RV et. al. A. Multivariate analysis of relationships between material properties, process parameters and tablet tensile strength for α-lactose monohydrate. Eur J Pharm Biopharm. 73(3);2009:424–31.

8.     Klevan I et. al. A statistical approach to evaluate the potential use of compression parameters for classification of pharmaceutical powder materials. Eur J Pharm Biopharm. 75(3);2010:425–35.

9.     Roopwani R and Buckner. Understanding deformation mechanisms during powder compaction using principal component analysis of compression data. Int J Pharm. 418(2);2011:227–34.

10.   Roopwani, Rahul et. al. Application of principal component analysis (PCA) to evaluating the deformation behaviors of pharmaceutical powders. Journal of Pharmaceutical Innovation. 8(2);2013:121-130.

11.   http://www.tau.ac.il/~rubi/PCA.pdf

12.   http://support.sas.com/publishing/pubcat/chaps/55129.pdf

 

 

 

Accepted on 16.06.2016        © RJPT All right reserved

Research J. Pharm. and Tech 2016; 9(8):1169-1173.